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Abstract 

The authors of the OPERA experiment [ArXiv: 1109.4897] claim that "the measurement indicates 
an early arrival time of CNGS muon neutrinos with respect to the one computed assuming the speed of 
light in vacuum!' 1 . In this note we analyze the statistical aspects of the experimental results presented 
in [ArXiv: 1109.4897], assuming that no hidden experimental bias exists. Due to statistical constraints, 
we show (through two different methods) that the experimental data presented in [ArXiv: 1109.4897] do 
not permit to conclude unambiguously with the existence of a superluminal behavior of neutrinos. The 
problem lies essentially in the interpretation of the data and not in their veracity. 
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1 Introduction 



The OPERA experiment attempts to measure the velocity of neutrinos using a one way method, 
the starting point being at the CERN and the end point being at the Laboratorio Nazionale 
del Gran Sasso (LNGS). This method needs an accurate knowledge of the distance baseline and 
a detailed budgeting of all delays with a very accurate synchronization of clocks at both ends 
of the baseline. The stated result, after correction of all systematic bias, is that the Time Of 
Flight (TOF) of the neutrinos is shorter by 60 db 6.9 (stat.) ± 7.4 (sys.) ns than that expected for 
a travel at the speed of light. 

A large number of papers dedicated to this article are already present on ArXiv [H O HI 
El El El El QUI HH H21 [131 HI], this list representing only few examples. They can be roughly 
divided into three categories. The papers that try to justify or invalidate the results of OPERA 
on a pure theoretical level, the comments on the latter that question the theoretical arguments 
developed, and the papers that question the experimental methodology of OPERA and the 
possible experimental biases (as clock synchronization |13|). Here we address the question of 
whether the stated result is consistent from a purely data analysis point of view (only using the 
informations given in p]). Moreover we assume that the experimental results presented in pQ 
do not contain any hidden systematic bias. 

In order to develop our analysis, we need first to recall briefly the experimental results obtained 
by OPERA and how these data have been treated to lead to the claim of a superluminal behavior 
of neutrinos. Then we apply our data analysis to address the question of whether a superluminal 
effect can be deduced from the data. 

2 A simplified presentation of OPERA results and their data 
analysis (with comments) 

For more details see pQ. 

2.1 The experiment and its framework 

The physical phenomenon observed by OPERA can be roughly seen as being the propagation of 
an energetic pulse transmitted by different particles over time. At the beginning the energetic 
pulse is a proton one (selected in energy) that is partially converted (by collision on a target) 
into a meson one (K or ir) which itself decays into a neutrino one. These steps occur at CERN. 
Then the neutrino pulse travels about 730km and is (partially) detected at LNGS. Moreover, 
because of the domain of energy involved, mesons and neutrinos can be assumed to travel at 
the speed of light (with a high precision). 

The initial proton pulse is characterized on the target by its instantaneous courant represented 
by the waveform w(t). The neutrinos detected at LNGS allow to rebuild in principle the in- 
stantaneous neutrino rate <f> v {t) (see the section below for the practical problems encountered). 
Then the simple relation expected is 

(f) v (t) = Kw(t- A) with A = L/c, (1) 

where K is some constant of proportionality and L is the distance traveled by the particles. 
In fact the situation is more complicated due to the low efficiency of neutrinos detection. Then 
an average between different experiments (different pulses) is needed, leading to the averaged 
quantities w(t) and 4> v (t) that are expected to verify the previous relation 

4> v (t) = Kw{t — A). (2) 
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The authors of pQ claim that this relation is not verified, but the following delayed one 



<j> v (t) = Kw(t-A-r). (3) 

is verified (after correction of all systematic bias) with the negative delay 

r = -60 ± 6.9 (stat.) ± 7.4 (sys.) , (4) 

stating an early arrival of neutrinos and then a superluminal neutrino effect. So their claim is 
divided into two different assertions: 

(a) The expected delayed proportionality rule ([2]) is false, 

(b) The new delayed relation §3§ is true. 

Since there is no direct logical inference between these two assertions, let us remark on a pure 
logical level that (a) can be true, (b) being false. In other words it is possible that the relation 
between w(t) and 4> v (t) is not at all a simple delayed rule of proportionality. 

2.2 The data 

2.2.1 The averaged proton waveforms 

The averaged waveform w(t) is given in [lj (fig. 9) which is here reproduced in figHJ In fact 
two different waveforms W\{t) and u>2(t) are given corresponding to two extractions that is two 
successive proton pulses. Each pulse Wi(t) has an approximate rectangular shape and a duration 
of about 10 /xs (for a stated superluminal effect of about 60 ns). 
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Figure 1: The averaged waveforms wi(t) and W2(t) given in pQ (fig. 9). 



2.2.2 The detected rate of neutrinos 

First it must be noticed that if the detected neutrinos at LNGS are time-stamped, their exact 
creation (in time and position) at CERN is unknown. Therefore it is impossible to measure 
the individual velocity of each neutrino and only a procedure dealing with the whole pulse is 
possible. 

Furthermore the efficiency of the neutrino detection being very low, the number of events de- 
tected per unit of time is small, as a consequence it is not easy to rebuild with precision the 

instantaneous neutrino rate 4> v {t) oc — event (t). Indeed this procedure requires to count the 
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number of events AN event during the interval of time At and this number AN event must be suffi- 
cient to have a relative uncertainty (due to statistical effects) V AiV cvcnt / AN cvcnt = 1/ \J AiV cvcnt 
small enough. Therefore At should not be too small. Of course this introduces an uncertainty 
on time, or a time resolution, typically ±At/2, since this procedure gives only access to the 
averaged rate during At. 

In fact two approximations of 4>u\t), (i = 1,2) are presented in PQ: in fig. 11 of [lj with 
At = 150 ns, and in fig.12 of [lj with At = 50 ns. 

The figure 11 of [1] (partially reproduced here in figE]) shows the data points of 4>v\t) and the 
waveforms v)i(t — A) (normalized), shifted or not by a delay. But this figure might be misleading 
because: 

• The graphs labelled with no delay (where Wi(t — A) and the <pi . (t) data points seem to 
differ by a shift) are in fact polluted by a (known) systematic bias on time of 987 ns, 

• The graphs that perfectly match, labelled with a delay of 1048ns, would have matched as 
well with the delay of 987 ns corresponding uniquely to the correction of the systematic 
bias, 

• The difference 1048 — 987 = 61 ns is not here significant because the § v data points are 
obtained with the time resolution At = 150 ns. 

In fact figure 11 of [1] seems to prove that taking into account the systematic bias on time and 
the averaged knowledge of <p v (time resolution of At = 150 ns), the usual delayed proportionality 
rule ([2]) is compatible with the data (no superluminal effect). 

To go beyond this result we need a more accurate data analysis. This is done in [I] through a 
likelihood procedure which is analyzed in the following section. The fig.12 of [lj obtained with 
the time resolution At = 50 ns is also analyzed below. 
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Figure 2: The first extraction averaged waveform w\(t — A) (normalized; red curve) and the 
corresponding data points AA^ ev ent/At (with At = 150ns) given in pQ (fig. 11). The left figure 
(5t = ns) does not contain the shift due to a systematic time bias, the right one (St = 1048.5 ns) 
contains both the shift due to the systematic bias (987 ns) and the supplementary delay of 60 ns. 
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2.2.3 Proving the early arrival of neutrinos 

Before analyzing the procedure used by p], let us analyze, on a pure logical level, how the data 
obtained can prove unambiguously the early arrival of neutrinos. 

Because of the rectangular shape of both pulses Wi(t — A) and 4>u\t), and because of their long 
duration (10 /is), only the left (leading) edge and the right (trailing) edge of the pulses and their 
possible shifts contain (possibly) significant informations. But while an early decrease of the 
neutrino signal (trailing edge) can be possibly due to some unknown property of the proton 
beam inducing a relation between the proton courant and the neutrino rate different from a 
simple delayed rule of proportionality, an early beginning of the neutrino signal (leading edge) is 
the signature of some superluminal effect (at least using arguments of usual Physics). Therefore 
the only parts of the data that possibly contain an unambiguous information about an early 
arrival of neutrino are the leading edges of W{(t — A) and 4>u\t) (if it is possible to prove that 
they differ only by a shift). 

As a consequence, any numerical procedure built to test the delayed relation ([3]) and 
that deals with the whole data set contains possibly an hidden logical bias: the value of 
the obtained delay (and its uncertainty) can be in fact polluted by a deformation of the neutrino 
signal not due at all to a superluminal effect. 

Therefore a possible method is the development of a numerical test that looks for 

(a) on one hand a possible shift between the leading edges of Wi{t — A) and (j>£\t), 

(b) independently, on the other hand, a possible shift between the trailing edges of iBi(t — A) 
and <$\t). 

If both computations gives the same result with a good confidence interval, then it can be 
reasonable to think that the neutrino pulse presents a superluminal effect. If the procedure is 
unable to give the same delay and/or sufficient confidence intervals (essentially in the leading 
edge case), then no proof of a superluminal effect exists. 

The figure 12 of [T| reproduced here in fig.© shows the behavior of leading and trailing 
edges of the pulses. The time resolution for computing the neutrino rate was here chosen as 
At = 50 ns. This introduces an uncertainty on time of about ±25 ns, (a little) less than the 
stated effect of the supplementary delay (advance) of 60 ns. 

Therefore these data are a good starting point to test our ideas about data treatments, even 
if the full precision of time-stamped events cannot be used in this case and then the expected 
accuracy is (much) lower. This analysis is done in the following sections. 

Let us analyze now the procedure used in [l] to obtain OPERA result. It is based on the 
whole pulse data set, and then it is possibly polluted by some hidden bias as mentioned above. 
Nevertheless this method is interesting because it allows to get rid of the time resolution problem 
inherent in the determination of the neutrino rate and then the full precision of time-stamped 

events can be used. This method uses the fact — x P"(t) where p u (t) is the probability 
density (in time) of detecting a neutrino and then from eq. (j3]h p u (t) oc w(t — A — r). Assuming 
the independence of events the probability density p v (ti,t2, ■ ■ ■ , £jv) f° r detecting N neutrinos is 
then 

N 

pu(ti,t 2 , ■ ■ ■ ,t N ) oc J| w(t n - A - t). (5) 

71=1 

The likelihood method is a bayesian procedure. Given the detection times (t\ , t 2 ; , • • • ,tffl) 
they attempt to specify the probability law for the parameter r. Up to a normalization this 
distribution is the function r h-> p v {t\ > ^2 > • • • > 4v^) because the function p u (ti, t 2 , ■ ■ ■ , ijv) can 
be seen as a conditional probability. Therefore the most probable value of r can be computed 
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Figure 3: Reproduction of the fig. 12 of [T] showing the details of the leading and trailing edges 
of the pulses (waveform in read) for two successive extractions. The previous delay of 1048.5 ns 
including 987 ns of systematic bias and 60 ns of supplementary delay is used. The time resolution 
for the neutrino rate is At = 50 ns. 



as the maximum of this function. 

In the following section we reanalyze differently the results mentioned in figj3l 



3 Our data analysis of the leading and trailing edges 

In this section the waveforms u>i(t) are assumed to be normalized and already shifted by the 
(known) delay A = L/c. 

We digitalized the data of figE] to obtain the waveforms Wi(i) and the data points of the 
neutrinos rates 4>v\t) (for the leading and trailing edges of the first and second extractions). 
We also removed the supplementary delay of 60 ns from the digitalized functions to have access 
to the waveforms v)i(t) corrected from the systematic time bias, but without any supplementary 
shift. The drawings are represented in figJH 

From these data we developed two types of analysis. The first one is a standard x 2 method, the 
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Figure 4: Digitalization of the data of fig|3l The uncertainties are represented by grey rectangles. 
The red curves are the delayed waveforms of fig. ([3]) , while the blue curves are the same waveforms 
assuming only a travel at the speed of light. 

second one is more unusual, but very adapted to this situation. 



Remark and Notation 

In the following we remove the index relative to the first and second extractions, and the index 
i is always relative to the data points of the neutrino rate (points (ij, &)). 



3.1 A standard x 2 method 
3.1.1 The procedure 

In each case (leading or trailing edge, first or second extraction), assuming the rate model 
4>(t) = w(t — r), we can define a first Xo( T ) function as 

TV N 

xI(t) = w y.^- w ^- t » 2 with S I = E< ( 6 ) 

<t> t=i i=i 

where the {o^.} are the (statistical) uncertainties on {<pi}. 

Nevertheless this Xo( T ) function does not take into account the uncertainty on time induced by 
the averaged rate. So a better choice is the following x 2 ( r ) function 

1 N ( f u+s dt\ 2 

X 2 ( T )= £2 / W{t ~ T) Y5) ^ 2S = At = 50nS ( 7 ) 

o 2=1 ^ Jti—8 / 

corresponding to the time averaged rate model < <j> > it) = J^g w(u — t)—. 
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Before using the method, we want first to discuss which types of informations are deducible 
from the x 2 minimization. By nature this test can only help us to answer to the following 
question: 

Assuming the existence of random, gaussian noises (with known widths) around the model 
values, are the data points compatible with this model? 

If the minimization process leads to a minimal value Xmin = x( T min) < 1) our choice of the 
normalization factor implies that the model for r = r m ; n is effectively compatible with the 
data. But this is not sufficient to conclude because: 

(a) Any model function (or value of r) such that x — 1 i s a l so compatible with the data on 
a statistical point of view; then only the interval [t\ , T2] around r m i n such that x < 1 is 
meaningful, 

(b) If the uncertainties on are not exactly known the previous interval [ti,T2] can be 
notably modified, 

(c) If the uncertainties on the data points are large, the condition x < 1 can be verified for very 
different model functions (not reduced to a specific family of one-parameter functional). 
Then if we are not absolutely certain of the model function type, even a narrow interval 
[tx,T2] around r m ; n is not sufficient to validate the model function. 

With these limitations in mind, we can look at the results. 
3.1.2 The results 
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Figure 5: Drawings of x(r) of eq.(J7|). On the left side: x( T ) f° r the leading edges of the two 
extractions (red curve for the first extraction, green curve for the second one). On the right 
side: x( T ) f° r the trailing edges of the two extractions with the same color codes. The horizontal 
dashed lines corresponds to the values x = 1 and x = V2 In 2. 

The drawings of x( r ) f° r the different leading and trailing edges (for the two extractions) 
are represented in figj5j 

The parameters deduced from the minimization are summarized on the table [TJ 

• The values (T m i n ,Xmin) represent the values obtained from the minimization (for the four 
edges). All Xmin verify the required condition Xmin < 1- 

• We have also computed the widths At of the interval [ti,T2] such that x < 1 an d X 5= 
V2 In 2, the value V21n2 coming from the evaluation of uncertainties from the width at 
half- height (for a gaussian) rather from its parameter a. Each value of At can give an 
estimate of the uncertainty 5t = At/ 2 on the value of r. In the present case the obtained 
values are very different. 
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Table 1: Summary of the x 2 minimization corresponding to the figJSJ 



3.1.3 Discussion 

At first sight of minima r m ; n in table [TJ this procedure seems to confirm the model function and 
the stated result in [T] of an early arrival of neutrinos characterized by the delay r ~ —60 ns. 
Indeed the four values of r mm , computed independently, agree with this value. But the width Ar 
of the intervals such that x < 1 (Ar — 40ns) and x < \/2 In 2 (Ar ~ 120ns) gives a completely 
different picture: because of the high sensitivity to the data uncertainties, 6t = Ar/2 is possibly 
of the same order of magnitude as the sought shift. Therefore, even if we can suspect a real 
shift, it is impossible to affirm that it exists. This paradox cannot be solved on this level: it 
requires the development of a new tool. This is done in the next section. 

3.2 Testing the type of the model function 

In this section we develop a statistical method that allows to test directly the validity of the 
previous model function < <f> > (t) = J^f w(u — r) — , rather than looking uniquely for the best 
value of r. 



3.2.1 The procedure 

It is a matter of fact that in the case of edges the function w(t) is either increasing (leading 
edge), either decreasing (trailing edge). This property is unchanged using our averaged function 

w(t) = (8) 

We deduce that this function is invertible (in the edge domains of the pulses). Then if the rate 
verifies (f>(t) = w(t — r), we deduce 

w- 1 (<P(t))=t-r (9) 
We deduce that our data points {(ij, 4>i)}iLi must verify 

T = t i -W- l {<j> i ) (10) 

Therefore we have only to verify that the set of data {t{ = U — w~ 1 (4>i)}fL 1 is compatible with 
a unique constant. Verifying that a cloud of points is reasonably in the neighborhood of an 
horizontal line is very easy. Then independently of the value of r, this test allows to verify if 
the model function is reasonable, because each experimental point generates a possible value of 
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r. Furthermore all usual statistical tests can be tried on the cloud of points. 

To keep the time structure of the data we built the new set of time-indexed data points {(U, T{ = 

ti—w~ 1 {4>i))}f =l for the four sets (edges) of experimental results analyzed in the previous section. 

3.2.2 The results 

The results are illustrated in figiU 

Let us remark that some 4> data points are outside the range of the function w, so they cannot 
have a reciprocal image by w . In the same way the error bars in the cp domain are not always 
contained in the range of w, so taking the reciprocal image by w~ l , they are not anymore 
centered on the experimental points. 

A simple glance at figE] shows that the points of the leading edges do not seem to be aligned 
on a horizontal line that differs from the t-axis, whereas this condition is essential to conclude 
positively to a superluminal effect. 

T(t) = t-lF'[0(t)] T(t) = t-ft- 1 [(J(t)] 




Figure 6: Experimental points of figi3] and figiU transformed into the sets {(U,Ti = U — 
w^ 1 {(f>i)}iLi- On the left side: points from the leading edges (red: first extraction, green: 
second extraction). On the right side: points from the trailing edges, same color codes. 
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Table 2: Summary of the statistical computations on r from the data points of figEJ 

Since we have now a sequence of values of r, we can compute the mean value and the 
standard error (or uncertainty) in a usual statistical sense. The calculations have been made 
independently on the leading edges and on the trailing edges: the results are summarized on the 
tabled 
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3.2.3 Discussion 



It can be divided in two parts: 

(a) Leading edges: the high value of the ratio ; — T ^ ; ~ 1.8 implies that it is impossible to 

define a reliable non-zero value of r because of the too noisy data. Then the superluminal 
effect is not unambiguously validated and if we are conservative, the value r ~ remains 
the most reasonable choice. 

(b) Trailing edges: the value of the ratio — T — ~ 0.8 is compatible with a possible noisy 
real shift. 

These results can be also (qualitatively) visualized plotting the data set {( tj, w~ 1 ((j)i) )}, since 
according to the model u> -1 ((/>(i)) = t — r. The resulting plot is given in the figure (JT|). The 
points coming from the leading edges fit well the curve f(t) = t, while the points of the trailing 
edges fit well the curve f(t) = t + 102. This representation is also natural since points coming 
from successive extractions can be represented on the same time axis and compared with the 
same function f(t). 



w _1 [0(t)] (ns) vf'Hm] (ns) 




Fi gure 7: Experimental points of fig |3] and fig|4] transformed into the sets {(ti,w (<^i))}i=i- 
On the left side: points from the leading edges (red: first extraction, green: second extraction) 
and the fitting function f(t) = t. On the right side: points from the trailing edges (same color 
codes) and the fitting function f(t) = t + 102. 

Put together these results show at the very least that the experimental data do not permit 
to (completely) exclude one of the following solutions: 

• a very noisy (leading edge) early arrival of neutrinos, 

• a small deformation of the neutrino signal (with respect to the proton pulse) in the trailing 

edge, 

• a mixing of the previous solutions. 

Therefore, as long as the assumption of a neutrino pulse deformation cannot be ruled out, it 
is impossible to claim that "the measurement indicates an early arrival time of CNGS muon 
neutrinos with respect to the one computed assuming the speed of light in vacuum" . 

Let us finish this discussion by a remark on the possible benefit of a Monte Carlo simulation 
(followed by our data analysis). In fact no additional information can be obtained, regardless of 
the simulation type, if no additional experimental data are given (for example the distribution 
of protons energy in the trailing edge). Indeed, since the true physical phenomena involved are 
unspecified, and since the observed data are (at least) ambiguous, no simulation (with a good or 
bad agreement with the data) is able to exclude or confirm assumptions. If the agreement is bad, 
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this can be due to a partial description of the true physical phenomena, but a priori this does 
not exclude anything (mixed effect). If the agreement is good, since the data are ambiguous, 
this is not enough to prove that other physical assumptions would not give the same results. 

4 Conclusion 

The OPERA experiment is very impressive due to the very high precision of the data, the 
latter being obtained from a huge and elaborate instrumental setup. Nevertheless the claim 
of a superluminal propagation of the neutrino beam cannot be deduced unambiguously from 
their data. The reasons lie in: (a) the smallness of the sought effect, (b) the experimental 
uncertainties, (c) a special feature of the data set. Put together these circumstances lead easily 
to wrong conclusions. In fact our data analysis shows that the time distribution of the neutrino 
packet is more probably not a time translation of the proton one, but rather a slight deformation 
(trailing edge). At the very least this solution cannot be ruled out. 
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